Skip to content

feat(llmrec): opt-in LLM context size telemetry and aggregation (P3.5 phase A) - #426

Merged
BaaRai merged 4 commits into
mainfrom
feat/context-telemetry
Nov 20, 2026
Merged

BaaRai merged 4 commits into
mainfrom
feat/context-telemetry

Conversation

@BaaRai

@BaaRai BaaRai commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Summary

Opt-in, size-only telemetry for the assembled LLM request, so Phases B (bounded context) and C (cache prefix) can be evaluated against data instead of guesses. Default off; no prompt, compaction or model behavior changes.

  • llmrec: ARTEX_CONTEXT_TELEMETRY=1 measures the assembled CompletionRequest on Stream and Complete: system bytes split at DynamicBoundary (matching norma joinSystem trim/drop-empty/\n\n semantics), serialized tool-schema bytes and count, message bytes by role, tool_result subset and last-message bytes. No content is stored.
  • db: migration 0009_llm_context.sql adds llm_context; ComputeLLMContextShares attributes each call's real input_tokens to sections pro rata by bytes; LLMContextSummary returns per-(worker, model) p50/p90/p99/max input, cache-read ratio and top tasks.
  • server: authenticated GET /api/llm/context/summary?days=30&top=20&task=... (no UI in Phase A).
  • docs/context-telemetry.md: semantics, SQL, retention guidance, and a representative fixture report with the real built-in volatility per lane (worker per-intent suffix, planner constraints, custom-prompt {{.Now}} only).

Verification

  • go test -p 1 -count=1 ./llmrec ./db ./server with Postgres: green.
  • Independent adversarial check: system bytes now match norma joinSystem for padded/empty segments; flag off writes zero rows; no content sentinel reaches the table; migration 0009 applies on a pre-0009 database.
  • Merge-tree clean against main e10c2b7.

@BaaRai
BaaRai merged commit 3559be6 into main Nov 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant